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A study was performed to determine if a Design of Experiments (DOE)/Response Surface 
Methodology could be applied to on-orbit thermal analysis and produce a set of Response Surface 
Equations (RSE) that accurately predict vehicle temperatures. The study used an integrated 
thermal model of the International Space Station and the Orion Outer mold line model. Five 
separate factors were identified for study: yaw, pitch, roll, beta angle, and the environmental 
parameters. Twenty external Orion temperatures were selected as the responses. A DOE case 
matrix of 1 10 runs was developed. The data from these cases were analyzed to produce an RSE 
for each of the temperature responses. The initial agreement between the engineering data and the 
RSE predictions was encouraging, although many RSEs had large uncertainties on their 
predictions. Fourteen verification cases were developed to test the predictive powers of the RSEs. 
The verification showed mixed results with some RSE predicting temperatures matching the 
engineering data within the uncertainty bands, while others had very large errors. While this study 
to not irrefutably prove that the DOE/RSM approach can be applied to on-orbit thermal analysis, it 
does demonstrate that technique has the potential to predict temperatures. Additional work is 
needed to better identify the cases needed to produce the RSEs 


Nomenclature 


C x = Coefficient 

Xj = Variable 

a = Standard deviation 

i, j, k = Summation integers 


I. Introduction 

The goal of this study was to apply a Design of Experiments (DOE)/Response Surface Methodology to on-orbit 
thermal analysis and determine if the resulting Response Surface Equations (RSEs) could predict temperatures 
within a defined error band. As detailed thermal models continue to increase in complexity, the time required to run 
them also increases. This limits the number of runs that an analysis team can practically perform in a given design 
cycle. This study explored using a DOE to define a set of detailed model runs, from which a RSE could be created 
for various temperature responses. An RSE is a polynomial expression that can then be used to predict temperatures 
for a wide range of variable combinations. The RSE is then solved to determine a grouping of extreme high and low 
temperature predictions. The conditions which cause these temperatures can then be entered into the detailed 
thermal model for a more thorough analysis. 

The concept of the Design of Experiments is over a century old, but not well known by many engineers. The 
foundation of DOE is the statistical variation of variables, or factors, between their defined upper and lower limits 
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and the observation of the system response, or responses, to this variation. This paper will not attempt to give a full 
mathematical background for DOE, but there is published literature on the subject. [1]. Once the combinations of 
factors and responses have been obtained, there are several commercially available computer packages which 
perform a regression fit of the responses based on the interaction of the factors. These interactions range from linear 
variations with a single factor, up to w-level interactions of all identified factors. The regressions produce a series of 
coefficients which are then coupled to the appropriate terms to produce a polynomial Response Surface Equation. 
This RSE can then be used to predict a response for any combination of factor values, provided the values are within 
the defined factor limits. Extrapolation outside of the factor limit is not recommended. An example of a DOE/RSE 
implementation is shown in Figure 1. 

DOE has been used in thermal analysis previously as part of an aerobreaking study on the Mars Reconnaissance 
Orbiter (MRO) [2, 3,4]. The MRO analysts used DOE to produce a set of RSEs to predict solar array temperatures 
based on factors such as atmospheric density, drag pass duration, and material properties. The resulting thermal 
model was able to predict solar array temperatures for a wide combination of factors. The model results were 
compared against flight data and showed that only a few data points fell outside of the ±3 a error bands applied to 
the predictions. 
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Figure 1. Flowchart Example of DOE/RSE Implementation 


II. Orion and ISS Model Overview 

For this study, the authors made use of the Orion Outer Mold Line (OML) model, integrated with the simplified 
International Space Station (ISS) thermal model. The Orion OML was created by Lockheed Martin as part of the 
Constellation Program/Orion Project. This model is a simplified representation of the Orion spacecraft intended to 
perform screening analyses to locate hot/cold conditions in which the more detailed Orion Integrated Thermal 
Model can be run. The model’s main features are the radiators, solar array, propulsion modules, and crew module 
outer surface (see Figure 2). It consists of a geometric model for radiation heat transfer and lumped capacitance 
nodalized thermal model. 

The ISS thermal model used was the V6R1 ISS thermal model provided by Boeing, as shown in Figure 3. The 
geometric model is used to calculate radiation heat transfer and a lumped thermal capacitance nodalized model is 
used to calculated temperatures. Figure 4 shows the Orion OML model placed at the Forward Node 2 port location. 
The combined model contains approximately 3200 nodes. 
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Figure 2. Orion Outer Mold Line Model 



Figure 3. ISS Thermal Model 
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Figure 4. Orion Outer Mold Line Model Mated to the ISS thermal Model 


III. Orion/ISS RSE Development 

The first step in creating RSEs is to determine the factors to use in the DOE. For the Orion/ISS model, a natural 
choice for factors are the yaw, pitch, and roll, which define the vehicle’s on-orbit attitude, and the beta angle and 
natural environment, which determine the external heating. For this particular exercise, other variables such as the 
vehicle power levels were held constant. In the case of articulating surfaces, they were allowed to rotate per their 
defined algorithms, and were not locked in any particular position. Table 1 shows the selected variables, uncertainly 
values, low, nominal, and high limits. The uncertainty levels were based on an engineering estimate. 

Table 1. Orion/ISS Variable for Creating RSEs 


Variable 

Uncertainty 

Low Value 

Mid Value 

Upper Value 

Yaw 

10% 

-15° 

0° 

15° 

Pitch 

10% 

-20° 

-2.5° 

15° 

Roll 

10% 

-15° 

0° 

15° 

Beta Angle 

2% 

0° 

37.5° 

75° 

Environment 

5% 

-1 

0 

1 


The environment variable is a combination of the altitude, solar constant, albedo, and planetary infrared 
radiation. The hot/cold values for each of these values were normalized between a value of -1 and +1. Table 2 
contains the dimensional values for each of these parameters, taken from the Orion-to-ISS Interface Requirements 
Document [5]. The Hot Case values were used when the Environment variable was +1 and the Cold Case values 
when it was -1. Values were varied linearly between the upper and lower extremes. 


4 


Table 2. Environmental Constants 


Parameter 

Hot Case 

Cold Case 

Solar 

1423 W/m 2 

1321 W/m 2 

Albedo 

0.53 

0.20 

Planetary IR 

349 W/m" 

153 W/m' 

Altitude 

278 km 

460 km 


The next step was to determine which responses were to be measured from the model. Since this activity was a 
demonstration of the DOE/RSE approach, nodes were not selected with regard to temperature limits or sensitivity to 
the particular mission. Rather, the authors chose 20 nodes located at various points around the model. The locations 
of these points are show in Figure 5 and listed in Table 3. It should be noted that any of the model nodes could 
serve as a response and an RSE developed. In an actual design, the selected responses could be critical component 
temperatures, heater power, or any other model output of interest to the analyst. 

Table 3. Temperature Responses 


Node # 

(Submodel.###) 

Component 

Node # 

(Submodel.###) 

Component 

SOLARDEP.1014 

Solar Array, Sun Facing 

SR4D1. 30000 

Service Module 
Auxiliary Thrusters 

SOLARDEP.2014 

SR4D2. 30000 

SOLARDEP.il 14 

Solar Array, Space 
Facing 

SR4D3. 30000 

SOLARDEP.2114 

SR4D4. 30000 

SMCOMM.ll 

Service Module 
Communication 
Antennas 

SRCS 1.4000 

Service Module 
Reaction Control 
thrusters 

SMCOMM.21 

SRCS2. 60000 

SMCOMM.31 

SRCS3. 50000 

SMCOMM.41 

SRCS4.2000 

SMHGA.il 

High Gain Antenna 

SMSADA1.7042 

Solar Array Gimbal 
Mechanisms 

CMBKSHL.1004 

Crew Module Backshell 

SMSADA2.7002 


High Gain 
Antenna 
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Figure 5. Orion Outer Mold Line Model 


With the Factors and Responses defined, a DOE case matrix can be produced. Using the 5 factors and a face- 
centered central composite design, the DOE package recommended a set of 64 cases to be run. An additional 10 
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cases at the center point of each factor were run to help anchor the model. These runs are known as replicate runs. 
After some initial trials, it was decided to add an additional 36 cases to the matrix. This enabled the program to 
better fit the RSEs to the measured responses. In total, 1 10 runs were performed. One useful recommendation is to 
non-dimensonalize all variables to a -1 to +1 range. This allows the same DOE matrix to be used, even if the factor 
ranges change. The DOE case matrix was run in Thermal Desktop. For each of the 100 cases, a full radiation 
analysis for heating rates and radiation to space was conducted (25,000 rays shot for each radiation task) and the 
model was solved using a steady state solution solver. Therefore, the temperatures reported are orbit average 
temperatures. Note that if an analyst is interest in minimum or maximum temperatures, a transient run could be 
performed and the data scanned for the desired local min/max values. 

After completing the analysis runs with the detailed thermal model, the temperature data were entered back into 
the DOE software tool in order to produce the RSEs. One of the choices the analyst can make is the level of 
interactions to be considered. For example, a first order, or linear, regression would only produce a polynomial 
equation with an intercept and one term representing each of the factors (yaw, pitch, roll, beta angle, and 
environment in this case). This RSE would not serve as a good predictor for the response because it cannot account 
for the complexity of interacting factor effects. 


[ 1 ] 

Because of the complex nature of many engineering problems, the interaction of factors can play a significant 
role in predicting the response. For a problem of 5 factors, the maximum number of interactions would be a 5 th 
order polynomial. Flowever, a polynomial of that degree is not necessarily required. The DOE software performs 
an analysis of the data and recommends the highest level of interactions needed before the interaction effects 
become confounded, or aliased. For the current study, a cubic polynomial was recommended. This equation takes 
the form: 


[ 2 ] 


Each of the 20 separate temperature predictions will have its own unique RSE. Therefore, the DOE code will 
produce 20 RSEs, each with different coefficients for the interaction terms defined in equation 2 above. There are 
56 coefficients for each RSE with the cubic interactions. An example of the RSE for the temperature of node 
SOLARDEP.1014 is given in equation [3]. 


[3] 


Where: 


Y= Yaw, normalized between -1 and +1. 

P = Pitch, normalized between -1 and +1. 

R = Roll, normalized between -1 and +1. 

B = Beta Angle, normalized between -1 and +1. 

E= Environment, normalized between -1 and +1. 

One important consideration is the accuracy of the RSE. To establish a ±3 a value for the predictions, the RSE 
prediction was subtracted from the engineering model output. Assuming a normal distribution, a one a value can be 
found using statistics. The error values are shown in Table 4. One of the facts that leaps from the table is the wide 
spread in 3 ex values. Some of the variation is as low as 3.5 °F, while others range up to 47 °F. This implies that 
additional cases may be needed for those responses to better characterize the response surface. Also, it is doubtful 
that an uncertainty as large of 40 °F would be useful to a thermal analyst. Figure 6 shows that all of the engineering 
model data falls within the RSE prediction error band for the 100 DOE cases. This is not entirely unexpected since 
these cases were used to create the RSEs themselves. Nevertheless, it is useful to see that even for an RSE with a 
wide 3 (rvalue, the general agreement between the RSE prediction and the engineering model is within 20 °F. Note 
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that for perfect agreement, the data points would lie along the “truth line,” indicating that the RSE and Thermal 
Desktop models were predicting the same temperature. 


Table 4. Temperature Response 3 cr Values 


Node # 

(Submodel.###) 

3-cr Value 
(Deg F) 

Node # 

(Submodel.###) 

3-crValue 
(Deg F) 

SOLARDEP.1014 

25.0 

SR4D 1.30000 

38.6 

SOLARDEP.2014 

17.7 

SR4D2. 30000 

33.5 

SOLARDEP.il 14 

28.7 

SR4D3. 30000 

36.1 

SOLARDEP.2114 

13.1 

SR4D4. 30000 

39.1 

SMCOMM.ll 

14.0 

SRCS 1.4000 

32.0 

SMCOMM.21 

3.5 

SRCS2. 60000 

47.1 

SMCOMM.31 

16.9 

SRCS3. 50000 

38.2 

SMCOMM.41 

27.3 

SRCS4.2000 

10.2 

SMHGA.il 

11.1 

SMS AD A 1.7042 

4.1 

CMBKSHL.1004 

29.7 

SMSADA2.7002 

5.3 



TD Prediction (Deg F) 


Figure 6. Comparison of Engineering Data to RSE Prediction for Node SR4D4. 30000 

IV. Results 

With the RSEs now in hand, fourteen verification cases were run to test the predictive power of the RSEs. These 
cases were selected to provide a wide range of factor combinations that had not been run previously (see Table 5). 
The RSE predictions were complete within a matter of seconds. The Thermal Desktop runs took longer to complete. 
Like the previous runs, the Thermal Desktop runs performed a radiation analysis using 25,000 rays per node and a 
steady-state solution routine. One change was to shoot and additional 100,000 rays from the RCS and Auxiliary 
thruster submodels. This change was made based on feedback from an initial data review with the model developers 
that the additional rays would provide a better temperature response for those nodes. The Thermal Desktop model 
run took approximately 6 hours on a dual quad-core processor with 8GB of RAM. 
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Table 5. 14-Case Verification Matrix 
Note that all values are normalized. 


Case # 

Yaw 

Pitch 

Roll 

Beta 

Env 

1 

1.00 

1.00 

1.00 

1.00 

0.80 

2 

0.20 

0.60 

1.00 

0.40 

- 0.80 

3 

0.20 

0.20 

1.00 

1.00 

0.60 

4 

0.00 

0.80 

0.00 

1.00 

- 1.00 

5 

0.00 

0.60 

0.80 

0.20 

- 0.40 

6 

- 0.20 

0.20 

0.20 

0.00 

0.60 

7 

- 0.40 

0.80 

0.20 

0.00 

- 0.60 

8 

- 0.40 

0.40 

- 0.80 

0.20 

0.20 

9 

- 0.40 

0.40 

- 1.00 

- 0.20 

0.40 

10 

- 0.60 

0.80 

- 0.80 

0.60 

- 0.60 


The results show varying degrees of agreement. In some instances, both the RSE and Thermal Desktop 
predictions are nearly identical, while in other instances the predictions are off by up to 100 °F. It is obvious that a 
100 °F error is completely unacceptable. Flowever, in most cases, the disagreement is < 20 °F on average. Figure 7 
shows an example of one node where the RSE performed particularly well. Note that all but one of the cases lie 
within the ±3 <7 value. This is encouraging given the fact that the RSE predictions at the hot and cold extremes 
match quite well with the Thermal Desktop predictions. 



Figure 7. Comparison of Thermal Desktop Prediction to RSE Prediction for Node SMFIGA.il for 

the 14 Checkout Cases 


Figure 8 shows one of the cases with a large disagreement (~80 °F) for one of the data points. This particular 
node also has several instances where the Thermal Desktop prediction does not fall with the RSE predictions ±3 a 
value. 
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Figure 8. Comparison of Thermal Desktop Prediction to RSE Prediction for Node SMCOMM.41 

for the 14 Checkout Cases 

V. Conclusions & Additional Work 

The above work does not irrefutably prove that the DOE/RSM approach can be applied to on-orbit thermal 
analysis. However, it does show the potential that this technique can be used successfully. Further work is needed 
to refine the development of the RSEs to ensure that they are valid over the entire range of factor combinations. The 
initial agreement of the RSEs with the DOE data is very promising, although the lack of consistent agreement with 
the verification cases is disappointing. 

Preliminary reviews of this work reveal several areas for improvement and additional work. These include 
performing transient runs of the system to measure orbital minimum and maximum temperatures for an entire 
submodel rather than a specific node. This may lead to better RSE agreement. Also, a suggestion was made to 
shoot a larger number of rays to ensure the proper radiation heat exchange is accounted for. Finally, the integrated 
1SS and Orion models represent a very complex vehicle. Applying this technique to a simpler spacecraft design may 
also improve the RSE performance. 
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